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DETAILED ACTION 



Response to Amendment 



1. Applicants 1 amendment filed January 7, 2004, has been entered and made of 
record. 

2. Objection to the abstract of the disclosure is withdrawn in view of Applicants 1 
amendment. 

3. Applicants' arguments regarding Claims 2-26 have been fully considered but they 
are not persuasive. 

With regards to Claims 2 and 9, and Claims 6 and 7, in response to Applicants' 
request for providing documentary evidence concerning parallel processing of specified 
image subject extracting algorithms, Examiner refers the Applicants to the teachings of 
Ohmi et al (U.S. 5,923,779). Ohmi et al disclose a face recognition method and 
corresponding apparatus wherein the extracting algorithms for extraction of face parts 
are performed in parallel as depicted in Figure 2. Additionally, this prior art disclose that 
the parallel processing is performed after precedent stages of implementing noise 
removal algorithm and edge detection algorithm. Considering the teachings of Saber et 
al and Ohmi et al, Claims 2-7 and 9-14 are not patentable over the cited prior arts. 

Regarding Applicants 1 arguments concerning Claim 8, it is submitted that Claim 8 
recites the device for implementing the methodology recited in Claim 1 . The prior art of 
record (Saber et al) clearly disclose the device for performing the algorithms or 
methodology of its face recognition as indicated in Section 3 titled "results", and 
depicted in Figures 4 and 5. Consequently, analogous arguments presented for Claim 
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1 are also applicable to Claim 8. It is further emphasized that Applicants have not 
indicated any arguments concerning the limitations in original Claim 1. The change 
control unit and subject processing unit are inherent components of the device for 
performing methodology of Claim 1. As a result, Applicants' statement "the change 
control unit and the image subject extraction processing unit would be obvious in view 
of the prior art, would be merely a result of hindsight" is not relevant and is invalid. 

Concerning Claim 15, Saber clearly discloses the broad limitation of performing a 
vote in an N-dimensional space of an image characteristic quantity. An image is a 2- 
dimensional space of the pixels representative of "pixel intensities" or "image 
characteristic quantity". The symmetry-based cost function disclosed teaches the vote 
in 2-dimensional space (spatial space) of the image. Claim language does not specify a 
plurality of extraction areas. Additionally, the alternate language of the claim requires 
only one "image subject extracting algorithm". Consequently, Saber's teachings 
concerning voting for the eyes (comprising, C n \ C n 2 , C n 3 , C n 4 and C n 5 ) reads claim 
language. Furthermore, since the location of the tip of the nose and center of the mouth 
is dependent of the location of the eyes (which has been obtained based on the voting), 
it will be concluded that the voting has been performed for other extraction areas as 
well. Aggregation of votes has been clearly taught in Formula (10) wherein C n is the 
combination of the individual votes. Nowhere in the Office Action, the distance of eyes 
from an axis is referred to as an aggregation of votes. 

Regarding Applicants' arguments concerning Claim 16, as it was indicated in the 
response to Claim 15 arguments, Claim 16 broadly recites, "performing a vote in an N- 
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dimensional space of an image characteristic quantity". A reasonable interpretation of 
the claim language is associating the N-dimensional space to the image characteristic 
quantity. In this case, a response analogous to those presented for Claim 15 is 
applicable to Claim 16. Even if Applicants desire to associate the space to "vote" and 
implies a "voting space" (which is not properly recited in the claim and contradicts Claim 
18 limitations), the symmetry-based cost function (voting function) is a five-dimension 
vector of C n \ C n 2 , C n 3 , C n 4 and C n 5 , representative of a five-dimensional space. 

Regarding Applicants' arguments concerning Claim 17, any of the cost functions, 
e.g., C n \ may be considered a preferential frame. 

Regarding Applicants' arguments concerning Claim 18, the image of the face is 
inherently a two-dimensional space of pixel values. Additionally, Section 2.1 describes a 
three-dimensional color space of the image characteristic quantity. 

Regarding Applicants 1 arguments concerning Claim 19, finding the minimum of 
the cost function in the eye region on horizontal minor axis is the weighting value 
lowering processing (Page 657, Formula (8)). 

Regarding Applicants' arguments concerning Claim 20, assigning lower values 
(sometimes even zero value) of weighting factors to the extracted data having large or 
small size in comparison to the mean value of the observed population of the data 
(outlier data), will remove a remarkably large size or a remarkably small size from 
extraction data. Applicants are definitely aware of this well known concept, and it 
appears that it is not necessary to refer the Applicants to the fundamentals of statistics. 
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Concerning Claims 21-26, responses analogous to those presented for Claims 
15-20 are applicable to Claims 21-26. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

5. Claims 1 , 8, 27 and 40 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Nguyen et al (Segmentation, Grouping and feature Detection for Face Analysis; 
IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4). 

Regarding Claim 1, Nguyen et al disclose a method of extracting a specified 
image subject which successively implements a plurality of specified image subject 
extracting algorithms, comprising the steps of: 

implementing an extracting algorithm of a precedent stage under a 
predetermined extracting condition to obtain an extraction result (Figure 1, connected 
component labeling to create a silhouette and extracting a head image; Page 594, 
Section 2.2); 

changing an extracting condition of a subsequent stage so as to be adapted to 
the thus obtained extraction result (Figure 1, skin segmentation and face-orientation; 
Page 594, Section 2.2); and 
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implementing an extracting algorithm of said subsequent stage under the thus 
changed extracting condition (Figures 3-6, Pages 596-598; first and second columns in 
Figure 3), 

wherein said precedent stage comprises extracting a shape of specified image 
subject (Figure 2, extraction of silhouette and head image). 

With regards to Claim 8, arguments analogous to those presented for the 
corresponding limitations of this claim in Claim 1 are applicable to Claim 8. 

With regards to Claim 27, arguments analogous to those presented for the 
corresponding limitation of this claim in Claim 1 are applicable to Claim 27. 

Regarding Claim 40, Nguyen et al further disclose a method of extracting a 
specified image subject according to Claim 1 , wherein said subsequent stage comprises 
detecting a color or hue of the specified image subject (Section 1.3, Color-based 
segmentation, feature detection). 

6. Claims 15-26, 38 and 39 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Saber et al (Face Detection and Facial Feature Extraction Using Color, 
Shape and Symmetry-based Cost Functions; IEEE Proceedings on Pattern 
Recognition, ISBN: 1015-4651). 

Regarding Claim 15, Saber et al disclose a method of extracting a specified 
image subject, comprising the steps of: 

performing image subject extraction processing by a specified image subject 
extracting algorithm or algorithms for each extraction area (Figures 1-3; Pages 656-657, 
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Section 2, Sub-sections 2.1-2.4, skin/non-skin classification and shape classification 
algorithms); 

performing a vote in an N-dimensional space of an image characteristic quantity 
for each extraction area extracted by said specified image subject extracting algorithm 
or algorithms (Figures 1-3; Pages 656-657, Section 2, Sub-section 2.5, cost function 
(5)); and 

performing weighting of degree of certainty as a specified image subject based 
on an aggregation value of the vote within a section area for aggregation in said N- 
dimensional space (Figures 1-3; Pages 656-657, Section 2, Sub-section 2.5, cost 
function Formula (5), weighted combination of the cost function) Formula (10)). 

Regarding Claim 16, Saber et al further disclose the method of extracting the 
specified image subject according to claim 15, wherein said image subject extraction 
processing by said specified image subject extracting algorithm or algorithms is 
performed through dividing it into a plurality of stages (Figures 1-3; Pages 656-657, 
Section 2, Sub-section 2.5, plurality of stages for eye, nose and mouth localization, and 
further plurality of stages based on cost functions C n 1 , C n 2 , C n 3 for eye detection); and 

said image subject extraction processing in a subsequent stage is preferentially 
applied to an extraction area in which said aggregation value in the voting space of 
said image characteristic quantity exceeds a predetermined value (Figures 1-5; Pages 
656-658, Section 2, Sub-section 2.5, Thresholds ti and t 2 ). 

Regarding Claim 17, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein said specified image subject 
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extraction processing by said specified image subject extracting algorithm or algorithms 
is performed through dividing it into a plurality of stages (Figures 1-3; Pages 656-657, 
Section 2, Sub-section 2.5, plurality of stages for eye, nose and mouth localization); and 
said image subject extraction processing in a subsequent stage is preferentially applied 
to an extraction area corresponding to said section area for aggregation within a 
preferential frame in the voting space of said image characteristic quantity (Figures 1-5; 
Pages 656-657, Section 2, Sub-section 2.5). 

Regarding Claim 18, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein a combination of a plurality of 
image characteristic quantities selected from the group consisting of a position, size, 
direction or orientation of an extraction area and, a posture, density or color tint of an 
image subject is used as the N-dimensional space of said image characteristic quantity 
(Figures 1-5; Pages 656-657, Section 2, Sub-sections 2.1-2.5. Skin/non-skin 
classification and shape classification algorithms implement a plurality of image 
characteristic quantities consisting of position, size, direction or orientation of an 
extraction area (Face, Eye, Nose and Mouth) and, a posture, density or color tint of an 
image subject (Skin-non-skin color classification in YES color space).). 

Regarding Claim 1 9, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 15, wherein weighting value lowering 
processing is applied to a region within a predetermined area on a specific 
characteristic axis with respect to a neighborhood of the region, in which said 
aggregation value became large, in said N-dimensional characteristic stage (Figures 1- 
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5; Pages 656-657, Section 2, Sub-sections 2.4-2.5. Principal axes of the elliptical skin 
classified region in N-dimensional eigenspace. Eyes are located on a line which is 
parallel to the minor axis represented by the direction of the eigenvector corresponding 
to the smaller eigenvalues. Finding the minimum of the cost function in the eye region 
on horizontal minor axis is the weighting value lowering processing (Page 657, Formula 
(8))- 

Regarding Claim 20, it is a conventional methodology in statistical decision 
processing to remove a remarkably large size or a remarkably small size from extraction 
data to increase the accuracy and reliability of extracting the subjects for recognition or 
classification purposes (Official Notice). Assigning lower values (sometimes even zero 
value) of weighting factors to the extracted data having large or small size in 
comparison to the mean value of the observed population of the data (outlier data), will 
remove a remarkably large size or a remarkably small size from extraction data. 

With regards to Claim 21, arguments analogous to those presented for Claim 15 
are applicable to Claim 21 . 

With regards to Claim 22, arguments analogous to those presented for Claim 16 
are applicable to Claim 22. 

With regards to Claim 23, arguments analogous to those presented for Claim 17 
are applicable to Claim 23. 

With regards to Claim 24, arguments analogous to those presented for Claim 18 
are applicable to Claim 24. 
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With regards to Claim 25, arguments analogous to those presented for Claim 19 
are applicable to Claim 25. 

With regards to Claim 26, arguments analogous to those presented for Claim 20 
are applicable to Claim 26. 

Regarding claim 38, Saber further disclose a method of extracting a specified 
image subject according to Claim 15, wherein said vote comprises an aggregation of 
points indicative of the specified image subject (Section 2.5, centroids of the "holes" 
within the skin segmentation mask). 

With regards to Claim 39, arguments analogous to those presented for Claim 38 
are applicable to Claim 39. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 2-7, 9-14, and 32-35 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Saber et al (Face Detection and Facial Feature Extraction Using 
Color, Shape and Symmetry-based Cost Functions; IEEE Proceedings on Pattern 
Recognition, ISBN: 1015-4651) in view of Ohmi et al (U.S. 5,923,779). 

With regards to Claims 2 and 9, arguments analogous to those presented for 
Claim 1 are applicable to Claim 2 and 9. Saber et al disclose performing a plurality of 
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image subject extracting algorithm of the subsequent stage (Sub-section 2.5; Algorithms 
resulted in cost functions indicated by Formulas (5) through (9)). 

Saber et al do not explicitly disclose the plurality of image subject extracting 
algorithm of the subsequent stage being performed by parallel processing. 

Ohmi et al disclose a face recognition method and corresponding apparatus 
wherein the extracting algorithms for extraction of face parts are performed in parallel as 
depicted in Figure 2. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Saber et al invention according to the teachings of Ohmi 
et al to perform the plurality of image subject extracting algorithm of the subsequent 
stage by parallel processing because it is an extremely well known procedure routinely 
implemented in the art to expedite image processing. 

Regarding Claim 3, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are areas to be subjected to extraction processing when 
implementing said plurality of specified image subject extracting algorithms of said 
subsequent stage (Figures 1-5, Face Areas). 

Regarding Claim 4, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are types of extracting algorithms to be implemented at said 
subsequent stage (Figures 1-3, Pages 656-657, Sub-section 2.4, Shape classification 
algorithms for locating face, nose, mouth, etc.). 
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Regarding Claim 5, Saber et al further disclose the method of extracting the 
specified image subject according to Claim 2, wherein said respective extraction 
processing conditions are control parameters inside extracting algorithms to be 
implemented in said subsequent stage (Figures 2 and 3; Pages 656-657, Sub-sections 
2.4 and 2.5, mi, m 2 , eigenvalues of R, (Xi, X 2 ), etc.). 

With regards to Claim 6, arguments analogous to those presented for Claim 2 
concerning parallel processing are applicable to Claim 6. 

Saber et al further disclose the method of extracting the specified image subject 
according to Claim 2, wherein said plurality of specified image subject extracting 
algorithms to be implemented in said each stage are of same combination in said 
plurality of stages (Sub-section 2.5. Algorithms resulted in cost functions are indicated 
by Formulas (5) through (10), utilizing the same combination of defining the centroids 
and location of the holes (eyes).). 

With regards to Claim 7, arguments analogous to those presented for Claim 2 
concerning parallel processing are applicable to Claim 7. 

Saber et al further disclose the method of extracting the specified image subject 
according to Claim 2, wherein said plurality of specified image subject extracting 
algorithms to be implemented in said each stage are of different combination in said 
plurality of stages (Sub-section 2.5. Algorithms resulted in localization of the nose and 
mouth as depicted in Figure 3.). 

With regards to Claim 10, arguments analogous to those presented for Claim 3 
are applicable to Claim 10. 
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With regards to Claim 1 1 , arguments analogous to those presented for Claim 4 
are applicable to Claim 1 1 . 

With regards to Claim 12, arguments analogous to those presented for Claim 5 
are applicable to Claim 12. 

With regards to Claim 13, arguments analogous to those presented for Claim 6 
are applicable to Claim 1 3. 

With regards to Claim 14, arguments analogous to those presented for Claim 7 
are applicable to Claim 14. 

With regards to Claims 32 and 33, arguments analogous to those presented for 
Claims 2 and 9 corresponding to the teachings of Ohmi et al concerning parallel 
processing are applicable to Claims 32 and 33. 

With regards to Claims 34 and 35, combined teachings of Saber et al and Ohmi 
et al disclose the image subject extraction parallel processing comprises: 

Skin color extraction, face contour extraction, hair-on-head extraction, 
eye/nose/mouth/eyebrow extraction, body extraction (head box in Figure 2 of Ohmi et 
al), and non-background area extraction (Figures 4b and 5b of Saber et al). 
9. Claims 28 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nguyen et al (Segmentation, Grouping and feature Detection for Face Analysis; 
IEEE Proceedings in Computer Vision, ISBN: 0-8186-7190-4) in view of Hasegawa et al 
(Real-time Parallel and Cooperative Recognition of Facial Images for an Interactive 
Visual Human Interface; IEEE Paper ISBN: 1051-4651). 
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Regarding Claim 28, Nguyen et al do not explicitly disclose a method of 
extracting a specified image subject according to Claim 1 , wherein said predetermined 
extracting condition comprises electronic flash or backlight information. 

Hasegawa et al disclose a real-time face recognition system comprising 
extraction condition regarding backlight (Page 386, Section 5.1.1, day time and evening 
lighting conditions). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Nguyen et al invention according to the teachings of 
Hasegawa et al to consider electronic flash or backlight information as one of the 
feature extraction parameters because it will compensate for environmental or ambient 
brightness changes and will provide more accurate recognition results. 

With regards to Claim 29, arguments analogous to those presented for Claim 28 
are applicable to Claim 29. 

10. Claims 30 and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Saber et al (Face Detection and Facial Feature Extraction Using Color, Shape and 
Symmetry-based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 
1015-4651) in view of Ohmi et al (U.S. 5,923,779) and Hasegawa et al (Real-time 
Parallel and Cooperative Recognition of Facial Images for an Interactive Visual Human 
Interface; IEEE Paper ISBN: 1051-4651). 

With regards to Claims 30 and 31 , arguments analogous to those presented for 
Claim 28 are applicable to Claims 30 and 31 . 
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1 1 . Claims 36 and 37 are rejected under 35 U.S.C. 1 03(a) as being unpatentable 
over Saber et al (Face Detection and Facial Feature Extraction Using Color, Shape and 
Symmetry-based Cost Functions; IEEE Proceedings on Pattern Recognition, ISBN: 
1015-4651) further in view of Ohmi et al (U.S. 5,923,779) and Wurtz (Object 
Reconstruction Robust Under Transitions, Deformations, and Changes in Background", 
IEEE Transactions on Pattern Analysis and Machine Intelligence, ISBN: 0162-8828). 

Saber et al and Ohmi et al do not explicitly disclose the method of extracting a 
specified image subject wherein the extraction algorithms comprise different degrees of 
resolution. 

Wurtz discloses a face recognition comprising extraction algorithms at different 
degrees of resolution (Sections 2.1 and 2.2). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Saber et al and Ohmi et al combination according to the 
teachings of Wurtz to utilize extraction algorithms at different degrees of resolution 
because it will discard low level features due to a variation in background (Wurtz; 
Abstract and Section 2.4). 

Conclusion 

12. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Contact Information 
13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mehrdad Dastouri whose telephone number is (703) 
305-2438. The examiner can normally be reached on Monday to Friday from 8:00 a.m. 
to 4:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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